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Since the mean score for a sample composed of several 
subgroups can be viewed as the sum of the mean of each subgroup 
weighted by the proportional size of the subgroup, then the mean 
change in a time period — in this case, from 1972 to 1980 — is the sum 
of the differences between the means for each subgroup, with each 
mean weighted by its proportional size. Thus, the mean change for the 
total sample is influenced both by mean change in the score for each 
group — in this case, a test score — and by change in the proportional 
size of the group. A decline or gain in mean test scores has both a 
subgroup mean component and a population shift component. In 
addition, how much a particular subgroup may contribute to the total 
change can be shown to depend on what might be called its weighted 
distance from the original total mean. How these simple concepts can 
be used to decompose score gains or declines is demonstrated by means 
of data from the seniors in the National Longitudinal Study (NLS) 
High School Class of 1972 and High School and Beyond (HSB) 1980. The 
presentation should be of both methodological and substantive 
i nter est . (Author/LMO ) 
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Since the mean score for a sainple conposed of several subgroups 
can be viewed as the sum of the mean of each s\ibgroup weighted by the 
proportional size of the subgroup, then the mean change in a time period — 
in our case, from 1972 to 1980— is the sum of the differences between 
the means for each subgroup, with each mean weighted by its proportional 
size. Thus, the mean change for the total sample is influenced both by 
mean change in the score for each group-- in our case, a test score — and 
by change in the proportional size of the group. A decline or gain in 
mean test scores has both a subgroup mean component and a population 
shift coiTi)onent. In addition, how much a particular subgroup may 
contribute to the total change can be shown to depend on what might be 
called its weighted distance from the original total mean. How these 
simple concepts can be used to decompose score gains or declines was 
demonstrated by means of data from the seniors in NLS 72 and HSB 80. 
The presentation should be of both methodological and substantive interest. 
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Score Change Partitioning Analysis of NLS and HSB Test Data 

Thomas L. Hilton, Judith Pollack, Donald A. Rock 
Educational Testing Service 

To draw conclusions about why individuals change over time requires 
a tightly controlled experiment wherein all the factors that are not of 
Interest are held constant and only the hypothesized causes are allowed 
to chan(^e. Longitudinal studies* wherein the variables not of interest 
are held constant by statistical means* are a secondtbest substitute for 
controlled experiments* for at least the same individuals are measured 
at the beginning and at the end of the data collection. But drawing 
conclusions about causes of individual change from cross-sectional data 
Is i^ossible in any rigorous sense. Nevertheless* when one is confronted 
with questions of pressing national importance and only cross-sectional 
data are available* one does the best one can. In the present study this 
consisted of three approaches: (1) examination of changer in mean test 
scores and partitioning of the mean change into components of interest* 
(2) analysis of covariance partitioning* and (3) path analysis. 

The first technique describes the extent of the relationship between 
selected population and behavioral ivhlfts to score decline. This type of 
analysis provides considerable detail about how classifying an individual 
on one or two variables at a tins relates to test score changes between 
1972 and 1980. In addition* this methodology attempts to partition the 
total score change into that part that was due to population shifts in 
the classification variables and that part that was due to mean changes 
within the classification groupings. 
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This partitioning procedure » however » does not lend itself to 
evaluating the impact of any one given classification variable or set 
of classification variables on score changes while controlling for the 
effects of numerous confounding variables. The second procedure that was 
used attempts to look at the relative impact of selected blocks of 
variables on the 1972-1980 mean score changes both before and after 
controlling for other confounding blocks of variables. The four blocks 
of variables are (1) demographics (e.g.* race/ethnicity, sex); (2) 
student behavior and attitudes; (3) school characteristics; and (4) home 
educational support systems (e.g.« parental influences, parental education, 
etc.). This partitioning procedure uses a "step down- analysis of 
variables that form a block while controlling for the remaining blocks. 
A second step in this step down ANCOVA is the identification of the 
variables in each block that contribute the nost to that particular 
block *s net effect on mean score change. 

In a sense this method takes the multiplicity of findings from the 
first method and sorts them into logical sets or blocks and summarizes 
the!;* net impact on mean test score change « 

The above two methods are primarily exploratory and descriptive in 
nature. The third and final method contrasts path analysis models for 
the 1972 and 1980 cohorts teparately in an effort to shed light on what 
changes in processes might have occurred to account for both the overall 
and, as well, differential score decline. 
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PTtltionlng of wtmn acore changes > It is well known that the 
overall «ean for two or «ore subgroups can be viewed as the sua of the 
subgroup means when each subgroup »ean is weighted in accordance with its 
proportion of the total group. In other words* 



of each subsaople • 

Furtheraore » a decline or gain in some tine period — in the present 
case» from 1972 to 1980— is as follows: 



ThuSf the total aean can change as a result of either a change in 
the aean of a subgroup or a change in the proportional representation of 
that subgroup. We will refer to the first component as the group mean 
change or 6 and the second as the population shift change or To 
estimate the magnitude of these components* two different calculations 
are made. The first component is calculated by applying » for each 
subgroup* the group^s proportion in 1972 to the group^s mean in the 1980 
population and then summing over all subgroups. The result is the mean 
score in 1980 that would have been expected if each subgroup *s representation 
In the population had not changed but its mean had changed. The difference 
between this number and the observed 1980 mean is the "Change due to 
subgroup mean changes** or G. 




Where Xt total sample and p is proportional size 
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Similarly t the "Change due to population ahift" or P is calculated 
by applying, for each subgroup, the group's proportion in the 1980 
population to the group's aean score in 1972, and then summing over all 
subgroups and proceeding as vith G. 

The sua of P and G ts not equal to the total mean change for the 
grouping variable, :Tor there may be an intv^idction between the two 
components. This interaction component (I) is calculated simply by 
subtracting the sum of P and G from the total mean change. In most 
cases, the term is a negligibly small number, and even where It is not 
its meaning is so difficult to interpret that the authors have not 
attempted to do so. 

Moment of group mean change. When, in a given tine period, the 
means for two or more subgroups change and, in addition, each subgroup's 
proportion in the population changes, it is not obvious how much each 
subgroup may have contributed to the total mean change. For this reason, 
a fourth statistic was computed that is referred to as the moment of the 
subgroup mean (M) because of its similarity to the familiar moment or 
torque in physics. This is the difference between the group's "weighted 
distance" in 1980 and in 1972. This distance is computed as the product 
of the group's proportional representation in its own year times the 
deviation of the group's mean from the 1972 grand mean. The moment, M, 
or, more precisely, the change in moment may be expressed as 




where p and p are the proportions at time 2 and time 1, X|-p 
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is the total aean «t tiae 1, •^Tg^ "nd X, are the aubgroup aean at 
tine 2 and tiae 1. . The sum of the ^ M'a or the change in the weighted 
distances over all groups is the distance of the wholt population from 
the original «ean, i.e., the difference between the 1980 and 1972 grand 
nean. A large number in the "Partition due to group" colunm can be a 
result of a population change for a highly deviant group (an increased 
proportion for a low-scoring group or decreased representation of a 
high-scoring group) and/or a score decline for a group that is unusually 
large relative to other groups. Interpretation of the score partitions 
oust be in conjunction with score nean and proportion statistics, as well 
as the "Change due to population shift" index. 

In discussing the results for a particular grouping variable, e.g., 
race /ethnicity, we frequently will refer to the contribution of a 
particular category or subgroup, e.g.. White students, to the overall 
decline, judging from the change in the subgroup's aoaent. It should be 
understood in all cases that the apparent causation aay well be spurious; 
that is, the change in the variable In question aay be a reflection of a 
change in sone aore fundatoentally causative variable with which it is 
correlated. In the interest of brevity we usually will oait the proper 
qualifications. 

Sone comments on nean score .change . To assist the reader in 
interpreting nean score changes for • group froa 1972 to 1980, the 
following comments are offered. The mean score for a particular subgroup 
of the total sample can change froa 1972 to 1980 for at least four 
possible reasons: 
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!• The aembers of the category may in 1980 represent a ssaller 
■ore selected sasple of the population than in 1972. For exavple, 

if the dropout rate for a category of students was such higher in 
1980 than in 1972, then it is reasonable to expect either less score 
decline or even a score gain from 1972 to 1980 if the students dropping 
out represented a lower ability stratum of the population, as usually is 
the casQ. A substantial decrease in the size of the subgroup from 1972 to 
1980 relative to other subgroups would be a warning that the 1980 subgroup 
may have been influenced by differential attrition for the period. 

2. A difference in the mean score for a subgroup may reflect a 
change in the personal characteristics of the population of students 
represented by the subgroup category. Por example, in 1972 the category 
**Suburban students in academic programs'* may have been predominantly White 
students, whereas in 1980 the category may have included a significant 
fraction of non-White students* 

3. From 1972 to 1980 the classroom, school, or coiwunity environment 
may have changed in such a way as to change the behavior of the students 

at the two times. 

4. The meaning of the category may have changed. For example, in 
1972 the category **Less than 5 hours of homework" may have meant that the 
average category member did 4 hours of homework each week whereas in 

1980 the members of that category did an average of 2 hours of homework each 
week. In other words, tlie authors may have been only partially successful 
in holding certain independent variables constant by subdividing the 
total group into categories. Each of these possible interpretations are 
exemplified in the results reported in the next section. 
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Tabls 1 

HtMXi IRT Vocabulary Scores In 1972 ^Bd 1980 by S«c, vlth PTtltlonlng 



A. Mean Vocabulary Score 
Male 
Female 
Total 



Tear 



1972 



6.44 
6.67 
6.55 



1980 



5.90 
5.69 
5.79 



1980-1972 
Difference 

-.54 
-.98 
-.77 



B. Weighted N 
Male 
Female 
Total 



1,425,843 1,218,450 
1,433,577 1,355,969 
2,859,421 2,574,419 



C. Proportion of High School Senior Population 
Male .50 
Female .50 

S. Change Due to Population Shift (P) 

E. Expected Change vith No Population Shift (G) 

F. Total Change Due to Interaction (I) 

C. Partition Total Change Due to Group 
Male 
Female 



.47 
.53 



-207,393 
- 77,608 
-285,002 

-.03 
.03 



0.01 
-0.76 
-0.01 



33Z 
67% 
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UhU 2,1 

Difference* Between 72-80 Senior Teet Score Me«n« 
When Each Block Is Separately Held Constant 



VOCABULARY READING MATH 

OBSERVED MEAN DECLINE -85 ■ 1.04 1.04 

MEAN DECLINE OR INCREASE AFTER ADJUSTING FOR: 



DEMOGRAPHICS ONLY ... 


.63 


NET 
CHANGE 
.22 


.79 


NET 
CHANGE 
.25 


.60 


NET 
CHANGE 
.44 


STUDENT BEHAVIORS ONLY... 


.28 


.57 


.48 


.56 


.56 


.48 


SCHOOL CHARACTERISTICS ONLY ... 


.21 


.64 


.24 


.80 


.23 


.81 


HOME SUPPORT ONLY... 


1.06 


-.21 


1.31 


-.27 


1.45 


-.41 
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Asjotm icAM sirmttfCfi fot i972-ifio itn ioous ir snjcm itoco or nnmrm vauabus 



wocKS or mtABUS rottnmT uuno AMO/ot 

QOimiMITlM; TO iODU oiaiifi 



wuMzt or mtAiut tMAtmu num to 

AMO/ot iUlSTIIC tOOU DHailft 



•44 C 



•<0 ,35 .30 .25 .20 .15 .10 .05 0 -.05 -.10 -.15 -.20 -.25 -.30 -•3$ 

I I i i i » » I j I I 1 1 1 1 1 

fOCABOUIf 



.72 i .h ) 
.42 



.07 I 



.>24 i CTIPPft MMAVlOt 

,13 \ SCHOOL CMAlACmtSTlCt 



mm aprrotT I -.ii 



UADIM 



«0f I POt)_ 



SCMOL CHAMCmiSTK^: 



aorrort i -.12 



MAmniATtct 







•09 1 omo 


) fTUpemr wuviots 




\ tdnOL aiAKACRtlSTlcS 


1 1 1 1 1 1 1 1 


HOHK aiPPOiT 1 -.11 


1 11 1 1 I 



.40 .3S .30 .23 .20 .13 .10 .03 0 -.03 -.10 -.13 -.20 -.23 -.30 
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